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Executive Summary 

State University Construction Fund 
Performance Indicators 



Scope of Audit Performance indicator reporting is a way to present program results in 

a measurable and meaningful manner. This type of reporting is also 
known as Service Efforts and Accomplishments (SEA) reporting. The 
objective of SEA reporting, according to the Governmental Accounting 
Standards Board (GASB), is to provide management, elected officials and 
the public, with more complete information about a governmental entity's 
performance (i.e., results of operations) than is provided in traditional 
financial reports. Essentially, SEA indicators should indicate what was 
spent, what was accomplished and how efficiently and effectively funds 
were used, thereby improving accountability. 

The State University Construction Fund (SUCF) is responsible for 
implementing the State University of New York's Master Capital Plan in 
a timely manner, with quality design and workmanship and at a 
reasonable cost. The State University system relies on SUCF to 
rehabilitate its existing facilities and to build new facilities at various 
campuses throughout the State. The workload of projects undertaken by 
SUCF is predominantly (92 percent) rehabilitation of existing structures. 
In fiscal year 1992-93 SUCF had approximately 130 staff and spent 
about $177.8 million for project administration, design and construction. 

Our audit, which covered the period April 1, 1990 through December 
31, 1993, focused on the need for SEA indicators in the SUCF- Our 
audit addressed the following questions about SUCF: 

• To what extent does SUCF use and report SEA indicators, that 
address overall construction efficiency, economy, and effectiveness 
issues? 

• Is SUCF's construction project data reliable and how is it used by 
SUCF to manage construction activities? 



Audit Observations Nationwide, SEA reporting is receiving greater attention, although only 
and Conclusions a relatively small number of construction organizations are currently 

reporting such data. We found that generally, SUCF has processes to 
manage individual projects and the construction program in total. SUCF 
does not utilize performance measures or SEA indicators to report how 
well it performs in fulfilling its mission. We believe a formalized SEA 
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reporting system can be an effective management tool that would 
complement SUCF's existing construction management practices. It 
would also provide a vehicle for SUCF to effectively communicate the 
results of its efforts to elected officials, other interested parties and the 
citizens of the State. 

In any construction project, there are three items which need to be 
controlled: time, cost and quality. In providing oversight of construction 
projects, SUCF collects an extensive amount of time and cost data which 
it uses to track the progress of each project. SUCF's managers judge 
their overall process by assessing how well they secure the maximum 
level of appropriations, how well they match appropriations to identified 
; ejects and how closely the funded projects meet expenditure ceilings. 
In addition, SUCF managers indicated they control project quality by 
assuring adherence to sta ^dards throughout the design and construction 
phases. Further, management has stated that, to some degree, quality 
issues are addressed through direct communication with users, however, 
such data is not currently captured on SUCF's project management 
system. Nevertheless, we determined that SUCF's controls over the 
project data that is currently collected are good and that the data is 
reliable, (see pp. 3-6) 

We found the available time and cost data lends itself to the development 
of SEA indicators for the agency's key functions: planning, design and 
construction management. Reporting such indicators would improve 
public accountability by showing the results of SUCF's efforts to 
accomplish its mission. Although SUCF must ultimately decide which 
indicators fairly represent the results of its efforts, we used the data 
available to management to develop examples of indicators SUCF may 
report. Our examples cover topics such as change order activity, 
construction contract completion time, and bid competition. Using data 
that is already available to SUCF, we show how SEA indicators could 
be applied to SUCF's administration of construction projects and used to 
report how effective SUCF is in fulfilling its mission, (see pp. 640) 

We believe SUCF could benefit by the use of SEA indicators. SUCF, 
in conjunction with other construction agencies, should identify key 
construction performance indicators and then provide for the collection 
of all necessary data. SUCF will then be in a position to provide more 
complete and therefore, more useful information to elected officials, 
other interested parties and the citizens of the State. In addition, by 
using SEA indicators, management will have a useful tool to better 
manage its projects, (see pp. 10-11) 



SUCF officials agree with our recommendations. SUCF management 
recognizes the potential benefits of developing and reporting SEA 
Indicators and indicated a willingness to consider this concept. Their 
response is attached in its entirety as Appendix B. 
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Introduction 



Background Performance indicator reporting is a way to present program results in 

a measurable and meaningful manner. This type of reporting is also 
known as Service Efforts and Accomplishments (SEA) reporting. The 
objective of SEA reporting, according to the Governmental Accounting 
Standards Board (GASB), is to provide management, elected officials and 
the public, with more complete information about a governmental entity's 
performance (i.e., results of operations) than is provided in traditional 
financial reports. Essentially, SEA indicators should indicate what was 
spent, what the citizens received for their money and how efficiently and 
effectively those funds were used. The GASB further states that SEA 
information should be relevant, understandable, comparable, timely, 
consistent and reliable. 

The State University Construction Fund (SUCF) is responsible for 
implementing the State University of New York's Master Capital Plan in 
a timely manner, with quality design and workmanship and at a 
reasonable cost. The State University system relies on SUCF to 
rehabilitate its existing facilities and to build new facilities at various 
campuses throughout the State. The workload of projects undertaken by 
SUCF is predominantly (92 percent) rehabilitation of existing structures. 
In fiscal year 1992-93, SUCF had approximately 130 staff and spent 
about $177.8 million for project administration, design and construction. 



Our audit focused on the need for service efforts and accomplishments 
indicators that address overall construction efficiency, economy, and 
effectiveness issues, and determined the extent to which SUCF reports 
such data. We also assessed the reliability of SUCF's construction 
project data and determined how that data is used to manage construction 
activities. Our audit covers the period April 1, 1990 through December 
31, 1993. 

To accomplish our objectives, we interviewed SUCF administrative, 
planning, design and construction managers, and contacted other 
governmental design and construction agencies through written question- 
naires and telephone surveys. We assessed the controls over the 
collection of project time and cost data and performed various tests we 
deemed necessary to determine whether the information provided is 
reliable. In addition, we reviewed relevant SUCF procedures, SUCF 
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records and various government reports related to performance indicators. 
A bibliography of these government reports is included as Appendix B. 

We conducted our audit in accordance with generally accepted 
government auditing standards. Such standards require that we plan and 
perform our audit to adequately assess those operations of SUCF which 
are included within the audit scope. Further, these standards require that 
we understand SUCF's internal control structure and compliance with 
those laws, rules and regulations that are relevant to SUCF's operations 
which are included in our audit scope. An audit includes examining, on 
a test basis, evidence supporting transactions recorded in the accounting 
and operating records and applying such other auditing procedures as we 
consider necessary in the circumstances. An audit also includes assessing 
the estimates, judgments, and decisions made by management. We 
believe that our audit provides a reasonable basis for our findings, 
conclusions and recommendations. 

We use a risk-based approach when selecting activities to be audited. 
This approach focuses our audit efforts on those operations that have 
been identified through a preliminary survey as having the greatest 
probability for needing improvement. Consequently, by design, finite 
audit resources are used to identify where and how improvements can be 
made. Thus, little audit effort is devoted to reviewing operations that 
may be relatively efficient or effective. As a result, our audit reports 
are prepared on an "exception basis." This report, therefore, highlights 
those areas needing improvement and does not address activities that may 
be functioning properly. 



Response of SUCF Draft copies of the matters contained in this report were provided to 
Officials to Audit SUCF officials for their review and comment. Their comments have 

been considered in preparing this report. 

Within 90 days after final release of this report, as required by Section 
170 of the Executive Law, the Chairman of the State University 
Construction Fund shall report to the Governor, the State Comptroller, 
and leaders of the Legislature and fiscal committees, advising what steps 
were taken to implement the recommendations contained herein, and 
where recommendations were not implemented, the reasons therefor. 
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Performance Indicators 



We observed that the reporting of achievements and program results 
nationwide is receiving greater attention; however, only a relatively small 
number of construction organizations are currently reporting such data. 
We found SUCF's capital process creates an extensive amount of data 
that is used to oversee projects and to manage activities (e.g., cost and 
time variances). We determined that SUCFs project data is reliable. 
Further, we found that generally SUCF has processes to manage 
individual projects and the program in total. SUCF does not utilize 
performance measures or SEA indicators to report how well it performs 
in fulfilling its mission. We believe a formalized performance measure- 
ment and reporting system can be an effective management tool that 
would complement SUCF's existing construction management practices 
and would provide a vehicle for SUCF to enhance accountability and to 
effectively communicate the results of its efforts to elected officials and 
the citizens of New York State. 



The GASB has researched and reported on the external reporting of 
selected SEA indicators. The GASB research team reported that their 
findings strongly suggest that the time has come for major experimenta- 
tion with implementing SEA measurements and with reporting SEA 
indicators to elected officials and the public. They further found that 
SEA information, where reported, has been of significant value to elected 
officials and citizens in assessing the results of government programs, 
leading to improvements ii efficiency and effectiveness in accomplishing 
program goals and objectives. The following are samples of some SEA- 
related activities we noted taking place at the Federal, state and local 
leveL 

Finding that waste and inefficiency in Federal programs undermine 
American confidence in government and that managers are disadvantaged 
in their efforts to improve program efficiency and effectiveness, because 
of insufficient articulation of program goals, the Congress recently passed 
the Government Performance and Results Act of 1993 (GPRA). The 
GPRA requires that Federal agencies: 

• develop strategic plans for program activities; 

• prepare annual performance plans covering each program activity, 
including performance goals expressed in objective, quantifiable and 
measurable forms; and 
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• report annually, actual program performance foDthe previous year. 



These requirements are intended to improve program effectiveness and 
public accountability by holding Federal agencies accountable for 
achieving program results. 

Several states have begun experimenting with using SEA indicators to 
facilitate budgetary decision making. The following example is useful 
in that it shows obstacles may be encountered in the development and 
implementation of SEA indicators. The United States General 
Accounting Office (GAO) issued a report in February 1993 entitled 
"Performance Budgeting • State Experiences and Implications for the 
Federal Government/ This report resulted from a study of five states 
(Connecticut, Hawaii, Iowa, Louisiana and North Carolina) regarded as 
leaders in performance budgeting. The objectives of the GAO study 
were to determine whether the selected states used performance measures 
for budget decision making, and to determine the potential implications 
of strte experiences for Federal efforts to institute performance 
budgeting. The study found that resource allocations continue to be 
driven, for the most part, by traditional budgeting practices. State 
officials encountered difficulties in achieving consensus on meaningful 
performance measures, noted dissimilarities in program and fund 
reporting structures, and acknowledged the limitations of current 
accounting systems. According to the report, officials at the Federal 
level anticipate similar problems. Overall, the GAO report observed that 
there was a strong commitment, among the state and Federal officials 
interviewed, to improve performance measurement within the public 
sector. Although these states are moving in the direction of SEA 
reporting, change will not likely happen overnight. 

We obtained a Portland, Oregon annual report which shows, for six 
major programs (e.g. police, fire, parks and recreation, etc.), spending, 
workload and results. The report compares fiscal year accomplishments 
to the prior 'hree years and to planned goals and performance objectives. 
It also rampares Portland's spending, staff levels, and workload to six 
similar cities. In addition, Portland surveyed its citizens to determine 
how they felt about the quality of city services. It appears Portland is 
one of the leaders in SEA reporting. 

On December 31, 1992, Governor Cuomo ordered the creation of a 
Statewide Quality through Participation Initiative. That initiative includes 
a provision for continuous improvement. We believe the adoption of 
SEA indicator reporting would help agency efforts toward fulfilling this 
initiative. The executive order explains continuous improvement as, "A 
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government where consistent evaluation and improvement, using key 
quality criteria, are built into all operations, Individuals and work 
groups will routinely assess and refine programs, procedures, and 
policies in the interest of better meeting the public's changing require- 
ments and standards of excellence/ 



SUCF's 

Const' ti >n Data 
Collects a and 
Accomplishment 
Reporting 



The construction process for every project includes three primary phases; 
planning, design and construction. In any construction project there are 
three items which need to be controlled: time, cost and quality. In 
providing oversight of construction projects, SUCF collects an extensive 
amount of time and cost data which it uses to track the progress of each 
project. However, we noted SUCF does not have adequate data on 
quality. 

SUCF's Office of Project Control and Initiation is primarily responsible 
for ensuring th?t all requirements to start project design are in place 
including: project viability, project budget, cost estimates, design 
contract, and resource allocation. 

SUCF's Office of Design is responsible for administering and supervising 
the preparation of design drawings and specifications for all SUCF 
projects, determining environmental impact and selecting the design 
consultant. This office monitors project progress through routine 
coordinator reports that are updated monthly. Review most often occurs 
on an individual project basis, and available data is usually not ag- 
gregated. 

SUCF's Office of Construction Management is responsible for con- 
struction execution from construction contract award through project 
completion. The goal of this office is to deliver completed projects 
according to design and contract on time and with quality field construc- 
tion. This office reviews change order activity on both a project level 
and an overall summary level. Further, this office routinely reviews 
timeliness by comparing payment percentages to elapsed time on a 
project-by-project basis. 

SUCF's managers judge their overall process by assessing how well they 
secure the maximum level of appropriations, how well they match 
appropriations to identified projects and how closely the funded projects 
meet expenditure ceilings. 

3CJCF managers indicated they control project qualitv by assuring 
adherence to design standards throughout the design and construction 
phases. Further, management has stated that, to some degree, quality 
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issues are addressed through direct communication with users, however, 
such data is not currently captured on SUCF's project management 
system. SUCF has not developed formal parameters to s^soss construc- 
tion quality. 



Project Data Is 
Reliable 



We obtained project data from SUCF for all constiuction contracts that 
had activity on or after April 1, 1990 through June 30, 1993. We 
assessed the controls over the collection of project time and cost data and 
performed various tests we deemed necessary to determine whether the 
information provided is reliable, including a comparison with similar data 
from the State Accounting System and verification of selected data 
elements, for a sample of projects, to source documents. We determined 
controls over data collection are good and construction project data is 
reliable. 



Reporting the 
Results of SUCFs 
Efforts 



We found the available time and cost data lends itself to the development 
of objective performance indicators for the agency's key functions: 
planning, design and construction management. Reporting such data 
would enhance public accountability by showing the results of SUCFs 
efforts to accomplish its mission. 

SUCF must ultimately decide which indicators fairly represent the results 
of its efforts. However, we used the data currently available to 
management to develop examples of indicators SUCF may report. 

We found that GASB research indicates there are three performance 
measurement categories: those that measure service efforts, those that 
measure service accomplishments, and those that relate service efforts to 
accomplishments. 

Service efforts (input indicators) can be measured by the amount of 
financial and non-financial resources (i.e., money, personnel, equipment, 
materials, etc.) put into a program or process. We found that some of 
SUCF's current reporting practices begin to address SEA concepts. For 
example, SUCF provides fiscal year project costs (i.e., consultant design 
and construction costs) and operating costs (i.e., administrative, planning, 
SUCF design and project oversight costs) in its annual reports. This 
data provides interested parties with an understanding of the resources 
used by SUCF in pursuit of its mission. Table 1 contains a summary 
of SUCF project and operating costs for each of the three fiscal years 
ended March 31, 1991, 1992 and 1993. 
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TABLE 1 

Project and Operating Expenditures by Fiscal Year (1) 


Fiscal Year 
Ended March 
31 


Project Costs 


Operating 
Costs 


Total 


1991 


$106,284,689 


$8,350,3 '5 


$114,635,034 


1992 


133,275,886 7,750,381 


141,026,267 


1993 


169,299,230 


8,471,878 


177,771,108 


Total 


$408,859,805 


$24,572,604 


$433,432,409 


(1) As Reported in Audited Financial Statements. 



Service accomplishment indicators (output and outcome indicators) show 
what was provided and achieved with the resources used. Output 
indicators show the quantity of the service provided. Outcome indicators 
show accomplishments or results that occur because of services provided. 
SUCF could expand its reporting to include construction output 
indicators. For example, we developed a table (see table 2) showing the 
number of consultant design projects completed (i.e., sufficiently 
completed to enable occupancy) during the period April 1, 1990 through 
June 30, 1993. We grouped the projects by total costs, to give an 
indication of the size of these projects. This presentation is an indication 
of what SUCF accomplished with the resources it used. 
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TABLE 2 

Projects Completed By Cost Classification (1) 
April 1, 1990 through June 30, 1993 



Total Cost 
Classification 


Number of 
FTojects 


Total 
Actual Cost (2) 


UP To $100,000 


19 


$ 1,140,700 


$100,000 To $500,000 


107 


28,061,400 


$500,000 To $1 
Million 


20 


13,766,500 


$1 To $5 Million 


19 


36,314,900 


Over $5 Million 


4 


42,854,300 


Grand Total 


169 


$122,137,800 



(1) Includes only those projects designed bj an outside 
consultant and completed during the period. 



(2) Includes actual costs or most recent -estimates. 



Performance indicators can further be useful in showing how, overall 
SUCF is doing to control project costs (i.e., results or outcome). For 
example, we developed the following two tables (see tables 3 and 4) that 
show SUCF's efforts have kept average project change order cost to a 
minimum (3.5 percent) and most (317) projects were completed on time 
or early. On average completion time was 6.3 percent under estimate. 



TABLE 3 
Change Order Activity (1) 





» of 
Projects 


Award 
Amount 


Construction 
Costs (2) 


Increase 
(Decrease) 


Percent 


Increases 


325 


$277,478,300 


$289,652,500 


$12,174,200 


4.4% 


No Change 


103 


16,636,100 


16,636,100 


0 


0.0% 


Decreases 


57 


29,857,800 


29,017,200 


(840,600) 


-2.8% 


Grand Total 


485 (3) 


$323,972,200 


$335,305,800 


$11,333,600 


3.5% 


(1) Includes in-house and consultant design projects completed during the audit period. 

(2) Includes actual costs or most recent estimates. 

(3) Includes 77 projects where the design and/or construction contract relates to several projects. 



TABLE 4 

Construction Contract Completion Time (1) 
Actual vs Estimate 





Number Of 
Projects 


Estimated 
Days 


Actual 
Days 


Delay 
Over/ 
(Under) 


Average 
Over/ 
(Under) 


Percent 
Over/ 
(Under) 


Delayed 


88 


31,684 


41,353 


9,669 


110 


30.5% 


On-Time 


122 


57,498 


57,498 


0 


0 


0.0% 


Early 


195 


75,385 


55,400 


(19,985) 


(102) 


-26.5% 


Total 


405 (2) 


164,567 


154,251 ' 


(10,316) 


(25) 


-6.3% 



(1) Includes in-house and consultant design projects completed during the audit period. 

(2) Data on timeliness was available for 405 projects. 



Indicators that relate efforts to accomplishments show the resources used 
per unit of output or outcome. An analysis of bid competition is an 
example of a measure that relates efforts to accomplishments. For 
example, the following chart (see table 5) shows that it appears increased 
competition (i.e., as a result of SUCF efforts) may result in decreased 
construction contract award amounts (i.e., accomplishments). In most 
instances (576), SUCF obtained three or more bids far each project and 
on average was successful in obtaining a construction contract totaling 
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6.5 percent below the design estimate of construction. This type of data 
may further be useful in assessing design manager success in estimating 
construction costs. 



TABLE 5 
Bid Competition (1) 
April 1, 1990 through June 30, 1993 










Difference 


Number 
Of Bids 


Number Of 
Projects 


Pre-Bid 
Estimate (2) 


Award 
Amount 


Amount 


Percent 


1 


6 


$ 639,000 


$ 625,200 


$ (13,800) 


-2.16% 


2 


22 


3,919,000 


3,734,900 


(184,100) 


-4.70% 


3-7 


281 


342,090,800 


334,843,100 


(7,247,700) 


-2.12% 


. 8-10 


115 


130,753,600 


122,371,700 


(8,381,900) 


-6.41% 


>10 


180 


162,564,600 


136,567,600 


(25,997,000) 


-15.99% 


Total 


604 


$639,967,000 


$598,142,500 


$(41,824,500) 


-6.54% 


(1) The required data elements were available for 604 projects ir the database. 

(2) The estimate of construction costs at the end of design. 



Care must be taken to accompany any reported indicators with sufficient 
explanation of the data presented and to allow the reader to clearly 
understand any limitations as to how the data should be used. For 
example, although SUCF has custody of a project from need 
identification through completion, project movement is dependent on 
funding. Projects can be delayed for years due to lack of funding prior 
to the start of design and prior to the start of construction. Although 
fulfillment of campus need runs from project identification through 
project completion, active SUCF management of the process is 
concentrated within the areas of design and construction. Therefore, data 
accompanying reported performance indicators should explain causes of 
delays beyond SUCF's control. 

We attempted to demonstrate, using the preceding examples, how SEA 
indicators could be applied to SUCF's administration of construction 
projects, and used to report how effective SUCF is in fulfilling its 
mission. Our examples were not meant to be all inclusive, but rather 
were intended to show there is a host of indicators SUCF may choose 
to report. SUCF needs to initiate discussions internally and with other 
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interested parties to further identify and perfect key construction 
performance indicators. 



Recommendations 

1. Work, with other construction agencies to identify key 
construction performance indicators. 

2. Assess the availability of the data required for the identified 
indicators and develop systems needed to collect the ap- 
propriate data. 

3. On at least an annual basis, publicly report the identified 
performance indicators. 



n 



Major Contributors to This Report 



Robert Blot 
David R.Hancox 
Henry Kagan 
Robert Lindholm 
Brian Lotz 
Donald Geary 
William OToole 
Mark Radley 
Michael Cantwell 
Paul Bachman 



ERJC 



STATE 
UNIVERSITY 




CONSTRUCTION FUND 

STATE UNIVERSITY PLAZA 



OFFICE OF THE STATE COMPTROLLER 

f:03ERT;i. AT 1 'ORE 



AUG 18 1994 

■ .■ ~ r./.uc:;c: 

F.:\ I rt.\w^LREPCST!KS 



August 15, 1994 



Mr. Robert H. Attmore 

Deputy Comptroller 

Division of Management Audit 

& Financial Reporting 
Office of the State Comptroller 
Gov. A. E. Smith State Office Bldg. 
Albany, New York 1223 6 

Dear Mr. Attmore: 

We are in basic agreement with the findings and recom- 
mendations made in Audit Report No. 94-S-24, State University 
Construction Fund: Performance Indicators. The Fund agrees 
that the concept of reporting performance accomplishments 
through the use of such indicators has the potential for 
meaningful presentation. 

Concerning Recommendation No. 1, the Fund has briefed the 
Council of Contracting Agencies on this initiative and recommends 
that the Office of the State Comptroller provide that group with 
a complete presentation. In regard to Recommendation Nos. 2 and 
3, the Fund will continue to review its data with the objective 
of developing Performance Indicator type reporting which can be 
published. 

In the meantime, we will continue to look forward to 
examples of such reporting that have been successfully 
implemented in peer situations. 



Sincerely, 




cc: Mr. Stephen Fletcher 
Mr. Raymond Wolfe 
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